The @preserve_version@ attribute on collections was originally designed to allow clients to persist a preexisting collection version. This forced clients to make 2 requests if the intention is to "make this set of changes in a new version that will be kept", so we have changed the semantics to do just that: When passing @preserve_version=true@ along with other collection updates, the current version is persisted and also the newly created one will be persisted on the next update.
+h3. System token requirements
+
+System services now log a warning at startup if any of the system tokens (@ManagementToken@, @SystemRootToken@, and @Collections.BlobSigningKey@) are less than 32 characters, or contain characters other than a-z, A-Z, and 0-9. After upgrading, run @arvados-server config-check@ and update your configuration file if needed to resolve any warnings.
+
+The @API.RailsSessionSecretToken@ configuration key has been removed. Delete this entry from your configuration file after upgrading.
+
h3. Centos7 Python 3 dependency upgraded to python3
Now that Python 3 is part of the base repository in CentOS 7, the Python 3 dependency for Centos7 Arvados packages was changed from SCL rh-python36 to python3.
<notextile>
<pre><code> SystemRootToken: <span class="userinput">"$system_root_token"</span>
ManagementToken: <span class="userinput">"$management_token"</span>
- API:
- RailsSessionSecretToken: <span class="userinput">"$rails_secret_token"</span>
Collections:
BlobSigningKey: <span class="userinput">"$blob_signing_key"</span>
</code></pre>
These secret tokens are used to authenticate messages between Arvados components.
* @SystemRootToken@ is used by Arvados system services to authenticate as the system (root) user when communicating with the API server.
* @ManagementToken@ is used to authenticate access to system metrics.
-* @API.RailsSessionSecretToken@ is used to sign session cookies.
* @Collections.BlobSigningKey@ is used to control access to Keep blocks.
Each token should be a string of at least 50 alphanumeric characters. You can generate a suitable token with the following command:
table(table table-bordered table-condensed).
|_. OS version|_. Command|
|Debian 10 ("buster")|<notextile><code><span class="userinput">echo "deb http://apt.arvados.org/buster buster main" | tee /etc/apt/sources.list.d/arvados.list</span></code></notextile>|
+|Ubuntu 20.04 ("focal")[1]|<notextile><code><span class="userinput">echo "deb http://apt.arvados.org/focal focal main" | tee /etc/apt/sources.list.d/arvados.list</span></code></notextile>|
|Ubuntu 18.04 ("bionic")[1]|<notextile><code><span class="userinput">echo "deb http://apt.arvados.org/bionic bionic main" | tee /etc/apt/sources.list.d/arvados.list</span></code></notextile>|
-|Ubuntu 16.04 ("xenial")[1]|<notextile><code><span class="userinput">echo "deb http://apt.arvados.org/xenial xenial main" | tee /etc/apt/sources.list.d/arvados.list</span></code></notextile>|
{% include 'notebox_begin' %}
Arvados uses SSL to encrypt communications. Its UI uses AJAX which will silently fail if the certificate is not valid or signed by an unknown Certification Authority.
-For this reason, the @arvados-formula@ has a helper state to create a root certificate to authorize Arvados services. The @provision.sh@ script will leave a copy of the generated CA's certificate (@arvados-snakeoil-ca.pem@) in the script's directory so ypu can add it to your workstation.
+For this reason, the @arvados-formula@ has a helper state to create a root certificate to authorize Arvados services. The @provision.sh@ script will leave a copy of the generated CA's certificate (@arvados-snakeoil-ca.pem@) in the script's directory so you can add it to your workstation.
Installing the root certificate into your web browser will prevent security errors when accessing Arvados services with your web browser.
Arvados uses SSL to encrypt communications. Its UI uses AJAX which will silently fail if the certificate is not valid or signed by an unknown Certification Authority.
-For this reason, the @arvados-formula@ has a helper state to create a root certificate to authorize Arvados services. The @provision.sh@ script will leave a copy of the generated CA's certificate (@arvados-snakeoil-ca.pem@) in the script's directory so ypu can add it to your workstation.
+For this reason, the @arvados-formula@ has a helper state to create a root certificate to authorize Arvados services. The @provision.sh@ script will leave a copy of the generated CA's certificate (@arvados-snakeoil-ca.pem@) in the script's directory so you can add it to your workstation.
Installing the root certificate into your web browser will prevent security errors when accessing Arvados services with your web browser.
if cluster.ManagementToken == "" {
cluster.ManagementToken = randomHexString(64)
}
- if cluster.API.RailsSessionSecretToken == "" {
- cluster.API.RailsSessionSecretToken = randomHexString(64)
- }
if cluster.Collections.BlobSigningKey == "" {
cluster.Collections.BlobSigningKey = randomHexString(64)
}
in := `
Clusters:
z1234:
- ManagementToken: xyzzy
- SystemRootToken: xyzzy
+ ManagementToken: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+ SystemRootToken: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
API:
MaxItemsPerResponse: 1234
+ Collections:
+ BlobSigningKey: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
PostgreSQL:
Connection:
sslmode: require
# serving a single incoming multi-cluster (federated) request.
MaxRequestAmplification: 4
- # RailsSessionSecretToken is a string of alphanumeric characters
- # used by Rails to sign session tokens. IMPORTANT: This is a
- # site secret. It should be at least 50 characters.
- RailsSessionSecretToken: ""
-
# Maximum wall clock time to spend handling an incoming request.
RequestTimeout: 5m
}
`)
cluster, err := testLoadLegacyConfig(content, "-legacy-keepweb-config", c)
- c.Check(err, check.IsNil)
+ c.Assert(err, check.IsNil)
c.Check(cluster.Services.Controller.ExternalURL, check.Equals, arvados.URL{Scheme: "https", Host: "example.com", Path: "/"})
c.Check(cluster.SystemRootToken, check.Equals, "abcdefg")
}
`)
cluster, err := testLoadLegacyConfig(content, "-legacy-keepweb-config", c)
- c.Check(err, check.IsNil)
+ c.Assert(err, check.IsNil)
// The resulting ManagementToken should be the one set up on the test server.
c.Check(cluster.ManagementToken, check.Equals, TestServerManagementToken)
}
content := []byte(fmtKeepproxyConfig("", true))
cluster, err := testLoadLegacyConfig(content, f, c)
- c.Check(err, check.IsNil)
- c.Check(cluster, check.NotNil)
+ c.Assert(err, check.IsNil)
+ c.Assert(cluster, check.NotNil)
c.Check(cluster.Services.Controller.ExternalURL, check.Equals, arvados.URL{Scheme: "https", Host: "example.com", Path: "/"})
c.Check(cluster.SystemRootToken, check.Equals, "abcdefg")
c.Check(cluster.ManagementToken, check.Equals, "xyzzy")
f := "-legacy-git-httpd-config"
cluster, err := testLoadLegacyConfig(content, f, c)
- c.Check(err, check.IsNil)
- c.Check(cluster, check.NotNil)
+ c.Assert(err, check.IsNil)
+ c.Assert(cluster, check.NotNil)
c.Check(cluster.Services.Controller.ExternalURL, check.Equals, arvados.URL{Scheme: "https", Host: "example.com", Path: "/"})
c.Check(cluster.SystemRootToken, check.Equals, "abcdefg")
c.Check(cluster.ManagementToken, check.Equals, "xyzzy")
}
`)
cluster, err := testLoadLegacyConfig(content, "-legacy-git-httpd-config", c)
- c.Check(err, check.IsNil)
+ c.Assert(err, check.IsNil)
// The resulting ManagementToken should be the one set up on the test server.
c.Check(cluster.ManagementToken, check.Equals, TestServerManagementToken)
}
content := []byte(fmtKeepBalanceConfig(""))
cluster, err := testLoadLegacyConfig(content, f, c)
- c.Check(err, check.IsNil)
- c.Check(cluster, check.NotNil)
+ c.Assert(err, check.IsNil)
+ c.Assert(cluster, check.NotNil)
c.Check(cluster.ManagementToken, check.Equals, "xyzzy")
c.Check(cluster.Services.Keepbalance.InternalURLs[arvados.URL{Host: ":80"}], check.Equals, arvados.ServiceInstance{})
c.Check(cluster.Collections.BalanceCollectionBuffers, check.Equals, 1000)
"API.MaxKeepBlobBuffers": false,
"API.MaxRequestAmplification": false,
"API.MaxRequestSize": true,
- "API.RailsSessionSecretToken": false,
"API.RequestTimeout": true,
"API.SendTimeout": true,
"API.WebsocketClientEventQueue": false,
# serving a single incoming multi-cluster (federated) request.
MaxRequestAmplification: 4
- # RailsSessionSecretToken is a string of alphanumeric characters
- # used by Rails to sign session tokens. IMPORTANT: This is a
- # site secret. It should be at least 50 characters.
- RailsSessionSecretToken: ""
-
# Maximum wall clock time to spend handling an incoming request.
RequestTimeout: 5m
"io"
"io/ioutil"
"os"
+ "regexp"
"strings"
"git.arvados.org/arvados.git/sdk/go/arvados"
// Check for known mistakes
for id, cc := range cfg.Clusters {
for _, err = range []error{
+ ldr.checkToken(fmt.Sprintf("Clusters.%s.ManagementToken", id), cc.ManagementToken),
+ ldr.checkToken(fmt.Sprintf("Clusters.%s.SystemRootToken", id), cc.SystemRootToken),
+ ldr.checkToken(fmt.Sprintf("Clusters.%s.Collections.BlobSigningKey", id), cc.Collections.BlobSigningKey),
checkKeyConflict(fmt.Sprintf("Clusters.%s.PostgreSQL.Connection", id), cc.PostgreSQL.Connection),
ldr.checkEmptyKeepstores(cc),
ldr.checkUnlistedKeepstores(cc),
return &cfg, nil
}
+var acceptableTokenRe = regexp.MustCompile(`^[a-zA-Z0-9]+$`)
+var acceptableTokenLength = 32
+
+func (ldr *Loader) checkToken(label, token string) error {
+ if token == "" {
+ ldr.Logger.Warnf("%s: secret token is not set (use %d+ random characters from a-z, A-Z, 0-9)", label, acceptableTokenLength)
+ } else if !acceptableTokenRe.MatchString(token) {
+ return fmt.Errorf("%s: unacceptable characters in token (only a-z, A-Z, 0-9 are acceptable)", label)
+ } else if len(token) < acceptableTokenLength {
+ ldr.Logger.Warnf("%s: token is too short (should be at least %d characters)", label, acceptableTokenLength)
+ }
+ return nil
+}
+
func checkKeyConflict(label string, m map[string]string) error {
saw := map[string]bool{}
for k := range m {
_, err := testLoader(c, `
Clusters:
zzzzz:
+ ManagementToken: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+ SystemRootToken: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+ Collections:
+ BlobSigningKey: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
postgresql: {}
BadKey: {}
Containers: {}
err = yaml.Unmarshal(buf, &loaded)
c.Assert(err, check.IsNil)
+ c.Check(logbuf.String(), check.Matches, `(?ms).*SystemRootToken: secret token is not set.*`)
+ c.Check(logbuf.String(), check.Matches, `(?ms).*ManagementToken: secret token is not set.*`)
+ c.Check(logbuf.String(), check.Matches, `(?ms).*Collections.BlobSigningKey: secret token is not set.*`)
+ logbuf.Reset()
loader.logExtraKeys(loaded, supplied, "")
c.Check(logbuf.String(), check.Equals, "")
}
var logbuf bytes.Buffer
logger := logrus.New()
logger.Out = &logbuf
- cfg, err := testLoader(c, `{"Clusters":{"zzzzz":{}}}`, &logbuf).Load()
+ cfg, err := testLoader(c, `
+Clusters:
+ zzzzz:
+ ManagementToken: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+ SystemRootToken: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+ Collections:
+ BlobSigningKey: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa`, &logbuf).Load()
c.Assert(err, check.IsNil)
yaml, err := yaml.Marshal(cfg)
c.Assert(err, check.IsNil)
c.Check(logbuf.String(), check.Equals, "")
}
+func (s *LoadSuite) TestUnacceptableTokens(c *check.C) {
+ for _, trial := range []struct {
+ short bool
+ configPath string
+ example string
+ }{
+ {false, "SystemRootToken", "SystemRootToken: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa_b_c"},
+ {false, "ManagementToken", "ManagementToken: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa b c"},
+ {false, "ManagementToken", "ManagementToken: \"$aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabc\""},
+ {false, "Collections.BlobSigningKey", "Collections: {BlobSigningKey: \"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa⛵\"}"},
+ {true, "SystemRootToken", "SystemRootToken: a_b_c"},
+ {true, "ManagementToken", "ManagementToken: a b c"},
+ {true, "ManagementToken", "ManagementToken: \"$abc\""},
+ {true, "Collections.BlobSigningKey", "Collections: {BlobSigningKey: \"⛵\"}"},
+ } {
+ c.Logf("trying bogus config: %s", trial.example)
+ _, err := testLoader(c, "Clusters:\n zzzzz:\n "+trial.example, nil).Load()
+ if trial.short {
+ c.Check(err, check.ErrorMatches, `Clusters.zzzzz.`+trial.configPath+`: unacceptable characters in token.*`)
+ } else {
+ c.Check(err, check.ErrorMatches, `Clusters.zzzzz.`+trial.configPath+`: unacceptable characters in token.*`)
+ }
+ }
+}
+
func (s *LoadSuite) TestPostgreSQLKeyConflict(c *check.C) {
_, err := testLoader(c, `
Clusters:
MaxKeepBlobBuffers int
MaxRequestAmplification int
MaxRequestSize int
- RailsSessionSecretToken string
RequestTimeout Duration
SendTimeout Duration
WebsocketClientEventQueue int
"SystemRootToken": auth_token('system_user'),
"API": {
"RequestTimeout": "30s",
- "RailsSessionSecretToken": "e24205c490ac07e028fd5f8a692dcb398bcd654eff1aef5f9fe6891994b18483",
},
"Login": {
"SSO": {
class TransactionFailedError < StandardError
end
- @@config = nil
@@debuglevel = 0
class << self
attr_accessor :debuglevel
@arvados_api_version = opts[:api_version] || 'v1'
- @arvados_api_host = opts[:api_host] ||
- config['ARVADOS_API_HOST'] or
- raise "#{$0}: no :api_host or ENV[ARVADOS_API_HOST] provided."
- @arvados_api_token = opts[:api_token] ||
- config['ARVADOS_API_TOKEN'] or
- raise "#{$0}: no :api_token or ENV[ARVADOS_API_TOKEN] provided."
+ @config = nil
+ [[:api_host, 'ARVADOS_API_HOST'],
+ [:api_token, 'ARVADOS_API_TOKEN']].each do |op, en|
+ if opts[op]
+ config[en] = opts[op]
+ end
+ if !config[en]
+ raise "#{$0}: no :#{op} or ENV[#{en}] provided."
+ end
+ end
if (opts[:suppress_ssl_warnings] or
%w(1 true yes).index(config['ARVADOS_API_HOST_INSECURE'].
# result looks like Arvados::A26949680::Job.
namespace_class.const_set classname, klass
- self.class.class_eval do
- define_method classname.underscore do
- klass
- end
+ self.define_singleton_method classname.underscore do
+ klass
end
end
end
def client
@client ||= Google::APIClient.
- new(:host => @arvados_api_host,
+ new(:host => config["ARVADOS_API_HOST"],
:application_name => @application_name,
:application_version => @application_version.to_s)
end
end
def config(config_file_path="~/.config/arvados/settings.conf")
- return @@config if @@config
+ return @config if @config
# Initialize config settings with environment variables.
config = {}
# Note: If we start using additional configuration settings from
# this file in the future, we might have to read the file anyway
# instead of returning here.
- return (@@config = config)
+ return (@config = config)
end
begin
debuglog "Ignoring error reading #{config_file_path}: #{e}", 0
end
- @@config = config
+ @config = config
end
class Model
:parameters => parameters,
:body_object => body,
:headers => {
- :authorization => 'OAuth2 '+arvados.config['ARVADOS_API_TOKEN']
+ :authorization => 'Bearer '+arvados.config['ARVADOS_API_TOKEN']
})
resp = JSON.parse result.body, :symbolize_names => true
if resp[:errors]
arvcfg.declare_config "API.MaxIndexDatabaseRead", Integer, :max_index_database_read
arvcfg.declare_config "API.MaxItemsPerResponse", Integer, :max_items_per_response
arvcfg.declare_config "API.AsyncPermissionsUpdateInterval", ActiveSupport::Duration, :async_permissions_update_interval
-arvcfg.declare_config "API.RailsSessionSecretToken", NonemptyString, :secret_token
arvcfg.declare_config "Users.AutoSetupNewUsers", Boolean, :auto_setup_new_users
arvcfg.declare_config "Users.AutoSetupNewUsersWithVmUUID", String, :auto_setup_new_users_with_vm_uuid
arvcfg.declare_config "Users.AutoSetupNewUsersWithRepository", Boolean, :auto_setup_new_users_with_repository
# Rails.configuration.API["Blah"]
ConfigLoader.copy_into_config $arvados_config, config
ConfigLoader.copy_into_config $remaining_config, config
- secrets.secret_key_base = $arvados_config["API"]["RailsSessionSecretToken"]
+
+ # We don't rely on cookies for authentication, so instead of
+ # requiring a signing key in config, we assign a new random one at
+ # startup.
+ secrets.secret_key_base = rand(1<<255).to_s(36)
end
password: ${database_pw}
dbname: arvados_${database_env}
client_encoding: utf8
- API:
- RailsSessionSecretToken: $secret_token
Collections:
BlobSigningKey: $blob_signing_key
DefaultReplication: 1
projects / arvados.git / commitdiff