Currently only reads @RemoteClusters@ from centralized configuration. Still requires component-specific configuration file.
+h2(#keepproxy). keepproxy
+
+The legacy keepproxy config (loaded from @/etc/arvados/keepproxy/keepproxy.yml@ or a different location specified via -legacy-keepproxy-config command line argument) takes precedence over the centralized config. After you migrate everything from the legacy config to the centralized config, you should delete @/etc/arvados/keepproxy/keepproxy.yml@ and stop using the -legacy-keepproxy-config argument.
+
h2. arvados-controller
Already uses centralized config exclusively. No migration needed.
So that older Arvados sites don't lose access to legacy records, the API has been converted to read-only. Creating and updating jobs (and related types job_task, pipeline_template and pipeline_instance) is disabled and much of the business logic related has been removed, along with various other code specific to the jobs API. Specifically, the following programs associated with the jobs API have been removed: @crunch-dispatch.rb@, @crunch-job@, @crunchrunner@, @arv-run-pipeline-instance@, @arv-run@.
+h4. Keepproxy configuration migration
+
+(feature "#14715":https://dev.arvados.org/issues/14715 ) Keepproxy can now be configured using the centralized config at @/etc/arvados/config.yml@. Configuration via individual command line arguments is no longer available and the @DisableGet@, @DisablePut@, and @PIDFile@ configuration options are no longer supported. If you are still using the legacy config and @DisableGet@ or @DisablePut@ are set to true or @PIDFile@ has a value, keepproxy will produce an error and fail to start. Please see "keepproxy's config migration guide":{{site.baseurl}}/admin/config-migration.html#keepproxy for more details.
+
h4. No longer stripping ':' from strings in serialized database columns
(bug "#15311":https://dev.arvados.org/issues/15311 ) Strings read from serialized columns in the database with a leading ':' would have the ':' stripped after loading the record. This behavior existed due to legacy serialization behavior which stored Ruby symbols with a leading ':'. Unfortunately this corrupted fields where the leading ":" was intentional. This behavior has been removed.
<notextile>
<pre><code>~$ <span class="userinput">keepproxy -h</span>
-...
-Usage: keepproxy [-config path/to/keepproxy.yml]
-...
+Usage of keepproxy:
+ -config file
+ Site configuration file (default may be overridden by setting an ARVADOS_CONFIG environment variable) (default "/etc/arvados/config.yml")
+ -dump-config
+ write current configuration to stdout and exit
+[...]
+ -version
+ print version information and exit.
</code></pre>
</notextile>
-h3. Create an API token for the Keepproxy server
-
-{% assign railscmd = "bundle exec ./script/get_anonymous_user_token.rb --get" %}
-{% assign railsout = "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" %}
-The Keepproxy server needs a token to talk to the API server. On the <strong>API server</strong>, use the following command to create the token. {% include 'install_rails_command' %}
-
-h3. Set up the Keepproxy service
+h3. Update the cluster config
-Install runit to supervise the keepproxy daemon. {% include 'install_runit' %}
-
-The run script for the keepproxy service should set the environment variables @ARVADOS_API_TOKEN@ (with the token you just generated), @ARVADOS_API_HOST@, and, if needed, @ARVADOS_API_HOST_INSECURE@. The core keepproxy command to run is:
+Edit the cluster config at @/etc/arvados/config.yml@ and set @Services.Keepproxy.ExternalURL@ and @Services.Keepproxy.InternalURLs@. Replace @zzzzz@ with your cluster id.
<notextile>
-<pre><code>ARVADOS_API_TOKEN=<span class="userinput">{{railsout}}</span> ARVADOS_API_HOST=<span class="userinput">uuid_prefix.your.domain</span> exec keepproxy
-</code></pre>
+<pre><code>Clusters:
+ <span class="userinput">uuid_prefix</span>:
+ Services:
+ <span class="userinput">Keepproxy:
+ ExternalURL: https://keep.uuid_prefix.your.domain
+ InternalURLs:
+ "http://localhost:25107": {}
+</span></code></pre>
</notextile>
h3. Set up a reverse proxy with SSL support
EOF</span>
</code></pre></notextile>
+h2. Run Keepproxy
+
+h3. Start the service (option 1: systemd)
+
+If your system does not use systemd, skip this section and follow the "runit instructions":#runit instead.
+
+If your system uses systemd, the keepproxy service should already be set up. Start it and check its status:
+
+<notextile>
+<pre><code>~$ <span class="userinput">sudo systemctl restart keepproxy</span>
+~$ <span class="userinput">sudo systemctl status keepproxy</span>
+● keepproxy.service - Arvados Keep Proxy
+ Loaded: loaded (/lib/systemd/system/keepproxy.service; enabled)
+ Active: active (running) since Tue 2019-07-23 09:33:47 EDT; 3 weeks 1 days ago
+ Docs: https://doc.arvados.org/
+ Main PID: 1150 (Keepproxy)
+ CGroup: /system.slice/keepproxy.service
+ └─1150 /usr/bin/keepproxy
+[...]
+</code></pre>
+</notextile>
+
+h3(#runit). Start the service (option 2: runit)
+
+Install runit to supervise the Keep-web daemon. {% include 'install_runit' %}
+
h3. Testing keepproxy
Log into a host that is on an external network from your private Arvados network. The host should be able to contact your keepproxy server (eg keep.$uuid_prefix.arvadosapi.com), but not your keepstore servers (eg keep[0-9].$uuid_prefix.arvadosapi.com).
WebsocketClientEventQueue: 64
WebsocketServerEventQueue: 4
+ # Timeout on requests to internal Keep services.
+ KeepServiceRequestTimeout: 15s
+
Users:
# Config parameters to automatically setup new users. If enabled,
# this users will be able to self-activate. Enable this if you want
return nil
}
+type oldKeepProxyConfig struct {
+ Client *arvados.Client
+ Listen *string
+ DisableGet *bool
+ DisablePut *bool
+ DefaultReplicas *int
+ Timeout *arvados.Duration
+ PIDFile *string
+ Debug *bool
+ ManagementToken *string
+}
+
+const defaultKeepproxyConfigPath = "/etc/arvados/keepproxy/keepproxy.yml"
+
+func (ldr *Loader) loadOldKeepproxyConfig(cfg *arvados.Config) error {
+ if ldr.KeepproxyPath == "" {
+ return nil
+ }
+ var oc oldKeepProxyConfig
+ err := ldr.loadOldConfigHelper("keepproxy", ldr.KeepproxyPath, &oc)
+ if os.IsNotExist(err) && ldr.KeepproxyPath == defaultKeepproxyConfigPath {
+ return nil
+ } else if err != nil {
+ return err
+ }
+
+ cluster, err := cfg.GetCluster("")
+ if err != nil {
+ return err
+ }
+
+ loadOldClientConfig(cluster, oc.Client)
+
+ if oc.Listen != nil {
+ cluster.Services.Keepproxy.InternalURLs[arvados.URL{Host: *oc.Listen}] = arvados.ServiceInstance{}
+ }
+ if oc.DefaultReplicas != nil {
+ cluster.Collections.DefaultReplication = *oc.DefaultReplicas
+ }
+ if oc.Timeout != nil {
+ cluster.API.KeepServiceRequestTimeout = *oc.Timeout
+ }
+ if oc.Debug != nil {
+ if *oc.Debug && cluster.SystemLogs.LogLevel != "debug" {
+ cluster.SystemLogs.LogLevel = "debug"
+ } else if !*oc.Debug && cluster.SystemLogs.LogLevel != "info" {
+ cluster.SystemLogs.LogLevel = "info"
+ }
+ }
+ if oc.ManagementToken != nil {
+ cluster.ManagementToken = *oc.ManagementToken
+ }
+
+ // The following legacy options are no longer supported. If they are set to
+ // true or PIDFile has a value, error out and notify the user
+ unsupportedEntry := func(cfgEntry string) error {
+ return fmt.Errorf("the keepproxy %s configuration option is no longer supported, please remove it from your configuration file", cfgEntry)
+ }
+ if oc.DisableGet != nil && *oc.DisableGet {
+ return unsupportedEntry("DisableGet")
+ }
+ if oc.DisablePut != nil && *oc.DisablePut {
+ return unsupportedEntry("DisablePut")
+ }
+ if oc.PIDFile != nil && *oc.PIDFile != "" {
+ return unsupportedEntry("PIDFile")
+ }
+
+ cfg.Clusters[cluster.ClusterID] = *cluster
+ return nil
+}
+
const defaultKeepWebConfigPath = "/etc/arvados/keep-web/keep-web.yml"
type oldKeepWebConfig struct {
import (
"flag"
+ "fmt"
"io/ioutil"
"os"
"time"
check "gopkg.in/check.v1"
)
+func testLoadLegacyConfig(content []byte, mungeFlag string, c *check.C) (*arvados.Cluster, error) {
+ tmpfile, err := ioutil.TempFile("", "example")
+ if err != nil {
+ return nil, err
+ }
+ defer os.Remove(tmpfile.Name())
+
+ if _, err := tmpfile.Write(content); err != nil {
+ return nil, err
+ }
+ if err := tmpfile.Close(); err != nil {
+ return nil, err
+ }
+ flags := flag.NewFlagSet("test", flag.ExitOnError)
+ ldr := testLoader(c, "Clusters: {zzzzz: {}}", nil)
+ ldr.SetupFlags(flags)
+ args := ldr.MungeLegacyConfigArgs(ldr.Logger, []string{"-config", tmpfile.Name()}, mungeFlag)
+ flags.Parse(args)
+ cfg, err := ldr.Load()
+ if err != nil {
+ return nil, err
+ }
+ cluster, err := cfg.GetCluster("")
+ if err != nil {
+ return nil, err
+ }
+ return cluster, nil
+}
+
func (s *LoadSuite) TestDeprecatedNodeProfilesToServices(c *check.C) {
hostname, err := os.Hostname()
c.Assert(err, check.IsNil)
"ManagementToken": "xyzzy"
}
`)
- tmpfile, err := ioutil.TempFile("", "example")
- if err != nil {
- c.Error(err)
- }
- defer os.Remove(tmpfile.Name())
-
- if _, err := tmpfile.Write(content); err != nil {
- c.Error(err)
- }
- if err := tmpfile.Close(); err != nil {
- c.Error(err)
- }
- flags := flag.NewFlagSet("keep-web", flag.ExitOnError)
- ldr := testLoader(c, "Clusters: {zzzzz: {}}", nil)
- ldr.SetupFlags(flags)
- args := ldr.MungeLegacyConfigArgs(ldr.Logger, []string{"-config", tmpfile.Name()}, "-legacy-keepweb-config")
- flags.Parse(args)
- cfg, err := ldr.Load()
- if err != nil {
- c.Error(err)
- }
- c.Check(cfg, check.NotNil)
- cluster, err := cfg.GetCluster("")
- if err != nil {
- c.Error(err)
- }
- c.Check(cluster, check.NotNil)
+ cluster, err := testLoadLegacyConfig(content, "-legacy-keepweb-config", c)
+ c.Check(err, check.IsNil)
c.Check(cluster.Services.Controller.ExternalURL, check.Equals, arvados.URL{Scheme: "https", Host: "example.com"})
c.Check(cluster.SystemRootToken, check.Equals, "abcdefg")
c.Check(cluster.Users.AnonymousUserToken, check.Equals, "anonusertoken")
c.Check(cluster.ManagementToken, check.Equals, "xyzzy")
}
+
+func (s *LoadSuite) TestLegacyKeepproxyConfig(c *check.C) {
+ f := "-legacy-keepproxy-config"
+ content := []byte(fmtKeepproxyConfig("", true))
+ cluster, err := testLoadLegacyConfig(content, f, c)
+
+ c.Check(err, check.IsNil)
+ c.Check(cluster, check.NotNil)
+ c.Check(cluster.Services.Controller.ExternalURL, check.Equals, arvados.URL{Scheme: "https", Host: "example.com"})
+ c.Check(cluster.SystemRootToken, check.Equals, "abcdefg")
+ c.Check(cluster.ManagementToken, check.Equals, "xyzzy")
+ c.Check(cluster.Services.Keepproxy.InternalURLs[arvados.URL{Host: ":80"}], check.Equals, arvados.ServiceInstance{})
+ c.Check(cluster.Collections.DefaultReplication, check.Equals, 0)
+ c.Check(cluster.API.KeepServiceRequestTimeout.String(), check.Equals, "15s")
+ c.Check(cluster.SystemLogs.LogLevel, check.Equals, "debug")
+
+ content = []byte(fmtKeepproxyConfig("", false))
+ cluster, err = testLoadLegacyConfig(content, f, c)
+ c.Check(cluster.SystemLogs.LogLevel, check.Equals, "info")
+
+ content = []byte(fmtKeepproxyConfig(`"DisableGet": true,`, true))
+ _, err = testLoadLegacyConfig(content, f, c)
+ c.Check(err, check.NotNil)
+
+ content = []byte(fmtKeepproxyConfig(`"DisablePut": true,`, true))
+ _, err = testLoadLegacyConfig(content, f, c)
+ c.Check(err, check.NotNil)
+
+ content = []byte(fmtKeepproxyConfig(`"PIDFile": "test",`, true))
+ _, err = testLoadLegacyConfig(content, f, c)
+ c.Check(err, check.NotNil)
+
+ content = []byte(fmtKeepproxyConfig(`"DisableGet": false, "DisablePut": false, "PIDFile": "",`, true))
+ _, err = testLoadLegacyConfig(content, f, c)
+ c.Check(err, check.IsNil)
+}
+
+func fmtKeepproxyConfig(param string, debugLog bool) string {
+ return fmt.Sprintf(`
+{
+ "Client": {
+ "Scheme": "",
+ "APIHost": "example.com",
+ "AuthToken": "abcdefg",
+ "Insecure": false
+ },
+ "Listen": ":80",
+ "DefaultReplicas": 0,
+ "Timeout": "15s",
+ "Debug": %t,
+ %s
+ "ManagementToken": "xyzzy"
+}
+`, debugLog, param)
+}
"API.WebsocketClientEventQueue": false,
"API.SendTimeout": true,
"API.WebsocketServerEventQueue": false,
+ "API.KeepServiceRequestTimeout": false,
"AuditLogs": false,
"AuditLogs.MaxAge": false,
"AuditLogs.MaxDeleteBatch": false,
WebsocketClientEventQueue: 64
WebsocketServerEventQueue: 4
+ # Timeout on requests to internal Keep services.
+ KeepServiceRequestTimeout: 15s
+
Users:
# Config parameters to automatically setup new users. If enabled,
# this users will be able to self-activate. Enable this if you want
KeepWebPath string
CrunchDispatchSlurmPath string
WebsocketPath string
+ KeepproxyPath string
configdata []byte
}
flagset.StringVar(&ldr.KeepWebPath, "legacy-keepweb-config", defaultKeepWebConfigPath, "Legacy keep-web configuration `file`")
flagset.StringVar(&ldr.CrunchDispatchSlurmPath, "legacy-crunch-dispatch-slurm-config", defaultCrunchDispatchSlurmConfigPath, "Legacy crunch-dispatch-slurm configuration `file`")
flagset.StringVar(&ldr.WebsocketPath, "legacy-ws-config", defaultWebsocketConfigPath, "Legacy arvados-ws configuration `file`")
+ flagset.StringVar(&ldr.KeepproxyPath, "legacy-keepproxy-config", defaultKeepproxyConfigPath, "Legacy keepproxy configuration `file`")
flagset.BoolVar(&ldr.SkipLegacy, "skip-legacy", false, "Don't load legacy config files")
}
if legacyConfigArg != "-legacy-keepweb-config" {
ldr.KeepWebPath = ""
}
+ if legacyConfigArg != "-legacy-keepproxy-config" {
+ ldr.KeepproxyPath = ""
+ }
return munged
}
ldr.loadOldKeepWebConfig(&cfg),
ldr.loadOldCrunchDispatchSlurmConfig(&cfg),
ldr.loadOldWebsocketConfig(&cfg),
+ ldr.loadOldKeepproxyConfig(&cfg),
} {
if err != nil {
return nil, err
SendTimeout Duration
WebsocketClientEventQueue int
WebsocketServerEventQueue int
+ KeepServiceRequestTimeout Duration
}
AuditLogs struct {
MaxAge Duration
env['ARVADOS_API_TOKEN'] = auth_token('anonymous')
logf = open(_logfilename('keepproxy'), 'a')
kp = subprocess.Popen(
- ['keepproxy',
- '-pid='+_pidfile('keepproxy'),
- '-listen=:{}'.format(port)],
- env=env, stdin=open('/dev/null'), stdout=logf, stderr=logf, close_fds=True)
+ ['keepproxy'], env=env, stdin=open('/dev/null'), stdout=logf, stderr=logf, close_fds=True)
+
+ with open(_pidfile('keepproxy'), 'w') as f:
+ f.write(str(kp.pid))
+ _wait_until_port_listens(port)
print("Using API %s token %s" % (os.environ['ARVADOS_API_HOST'], auth_token('admin')), file=sys.stdout)
api = arvados.api(
"syscall"
"time"
+ "git.curoverse.com/arvados.git/lib/config"
"git.curoverse.com/arvados.git/sdk/go/arvados"
"git.curoverse.com/arvados.git/sdk/go/arvadosclient"
- "git.curoverse.com/arvados.git/sdk/go/config"
"git.curoverse.com/arvados.git/sdk/go/health"
"git.curoverse.com/arvados.git/sdk/go/httpserver"
"git.curoverse.com/arvados.git/sdk/go/keepclient"
var version = "dev"
-type Config struct {
- Client arvados.Client
- Listen string
- DisableGet bool
- DisablePut bool
- DefaultReplicas int
- Timeout arvados.Duration
- PIDFile string
- Debug bool
- ManagementToken string
-}
-
-func DefaultConfig() *Config {
- return &Config{
- Listen: ":25107",
- Timeout: arvados.Duration(15 * time.Second),
- }
-}
-
var (
listener net.Listener
router http.Handler
const rfc3339NanoFixed = "2006-01-02T15:04:05.000000000Z07:00"
-func main() {
- log.SetFormatter(&log.JSONFormatter{
- TimestampFormat: rfc3339NanoFixed,
- })
+func configure(logger log.FieldLogger, args []string) (*arvados.Cluster, error) {
+ flags := flag.NewFlagSet(args[0], flag.ExitOnError)
- cfg := DefaultConfig()
+ dumpConfig := flags.Bool("dump-config", false, "write current configuration to stdout and exit")
+ getVersion := flags.Bool("version", false, "Print version information and exit.")
- flagset := flag.NewFlagSet("keepproxy", flag.ExitOnError)
- flagset.Usage = usage
+ loader := config.NewLoader(os.Stdin, logger)
+ loader.SetupFlags(flags)
- const deprecated = " (DEPRECATED -- use config file instead)"
- flagset.StringVar(&cfg.Listen, "listen", cfg.Listen, "Local port to listen on."+deprecated)
- flagset.BoolVar(&cfg.DisableGet, "no-get", cfg.DisableGet, "Disable GET operations."+deprecated)
- flagset.BoolVar(&cfg.DisablePut, "no-put", cfg.DisablePut, "Disable PUT operations."+deprecated)
- flagset.IntVar(&cfg.DefaultReplicas, "default-replicas", cfg.DefaultReplicas, "Default number of replicas to write if not specified by the client. If 0, use site default."+deprecated)
- flagset.StringVar(&cfg.PIDFile, "pid", cfg.PIDFile, "Path to write pid file."+deprecated)
- timeoutSeconds := flagset.Int("timeout", int(time.Duration(cfg.Timeout)/time.Second), "Timeout (in seconds) on requests to internal Keep services."+deprecated)
- flagset.StringVar(&cfg.ManagementToken, "management-token", cfg.ManagementToken, "Authorization token to be included in all health check requests.")
-
- var cfgPath string
- const defaultCfgPath = "/etc/arvados/keepproxy/keepproxy.yml"
- flagset.StringVar(&cfgPath, "config", defaultCfgPath, "Configuration file `path`")
- dumpConfig := flagset.Bool("dump-config", false, "write current configuration to stdout and exit")
- getVersion := flagset.Bool("version", false, "Print version information and exit.")
- flagset.Parse(os.Args[1:])
+ args = loader.MungeLegacyConfigArgs(logger, args[1:], "-legacy-keepproxy-config")
+ flags.Parse(args)
// Print version information if requested
if *getVersion {
fmt.Printf("keepproxy %s\n", version)
- return
+ return nil, nil
}
- err := config.LoadFile(cfg, cfgPath)
+ cfg, err := loader.Load()
if err != nil {
- h := os.Getenv("ARVADOS_API_HOST")
- t := os.Getenv("ARVADOS_API_TOKEN")
- if h == "" || t == "" || !os.IsNotExist(err) || cfgPath != defaultCfgPath {
- log.Fatal(err)
- }
- log.Print("DEPRECATED: No config file found, but ARVADOS_API_HOST and ARVADOS_API_TOKEN environment variables are set. Please use a config file instead.")
- cfg.Client.APIHost = h
- cfg.Client.AuthToken = t
- if regexp.MustCompile("^(?i:1|yes|true)$").MatchString(os.Getenv("ARVADOS_API_HOST_INSECURE")) {
- cfg.Client.Insecure = true
+ return nil, err
+ }
+ cluster, err := cfg.GetCluster("")
+ if err != nil {
+ return nil, err
+ }
+
+ if *dumpConfig {
+ out, err := yaml.Marshal(cfg)
+ if err != nil {
+ return nil, err
}
- if y, err := yaml.Marshal(cfg); err == nil && !*dumpConfig {
- log.Print("Current configuration:\n", string(y))
+ if _, err := os.Stdout.Write(out); err != nil {
+ return nil, err
}
- cfg.Timeout = arvados.Duration(time.Duration(*timeoutSeconds) * time.Second)
+ return nil, nil
}
+ return cluster, nil
+}
- if *dumpConfig {
- log.Fatal(config.DumpAndExit(cfg))
+func main() {
+ logger := log.New()
+ logger.Formatter = &log.JSONFormatter{
+ TimestampFormat: rfc3339NanoFixed,
+ }
+
+ cluster, err := configure(logger, os.Args)
+ if err != nil {
+ log.Fatal(err)
+ }
+ if cluster == nil {
+ return
}
log.Printf("keepproxy %s started", version)
- arv, err := arvadosclient.New(&cfg.Client)
+ if err := run(logger, cluster); err != nil {
+ log.Fatal(err)
+ }
+
+ log.Println("shutting down")
+}
+
+func run(logger log.FieldLogger, cluster *arvados.Cluster) error {
+ client, err := arvados.NewClientFromConfig(cluster)
+ if err != nil {
+ return err
+ }
+ client.AuthToken = cluster.SystemRootToken
+
+ arv, err := arvadosclient.New(client)
if err != nil {
- log.Fatalf("Error setting up arvados client %s", err.Error())
+ return fmt.Errorf("Error setting up arvados client %v", err)
}
- if cfg.Debug {
+ if cluster.SystemLogs.LogLevel == "debug" {
keepclient.DebugPrintf = log.Printf
}
kc, err := keepclient.MakeKeepClient(arv)
if err != nil {
- log.Fatalf("Error setting up keep client %s", err.Error())
+ return fmt.Errorf("Error setting up keep client %v", err)
}
keepclient.RefreshServiceDiscoveryOnSIGHUP()
- if cfg.PIDFile != "" {
- f, err := os.Create(cfg.PIDFile)
- if err != nil {
- log.Fatal(err)
- }
- defer f.Close()
- err = syscall.Flock(int(f.Fd()), syscall.LOCK_EX|syscall.LOCK_NB)
- if err != nil {
- log.Fatalf("flock(%s): %s", cfg.PIDFile, err)
- }
- defer os.Remove(cfg.PIDFile)
- err = f.Truncate(0)
- if err != nil {
- log.Fatalf("truncate(%s): %s", cfg.PIDFile, err)
- }
- _, err = fmt.Fprint(f, os.Getpid())
- if err != nil {
- log.Fatalf("write(%s): %s", cfg.PIDFile, err)
- }
- err = f.Sync()
- if err != nil {
- log.Fatalf("sync(%s): %s", cfg.PIDFile, err)
- }
+ if cluster.Collections.DefaultReplication > 0 {
+ kc.Want_replicas = cluster.Collections.DefaultReplication
}
- if cfg.DefaultReplicas > 0 {
- kc.Want_replicas = cfg.DefaultReplicas
+ var listen arvados.URL
+ for listen = range cluster.Services.Keepproxy.InternalURLs {
+ break
}
- listener, err = net.Listen("tcp", cfg.Listen)
- if err != nil {
- log.Fatalf("listen(%s): %s", cfg.Listen, err)
+ var lErr error
+ listener, lErr = net.Listen("tcp", listen.Host)
+ if lErr != nil {
+ return fmt.Errorf("listen(%s): %v", listen.Host, lErr)
}
+
if _, err := daemon.SdNotify(false, "READY=1"); err != nil {
log.Printf("Error notifying init daemon: %v", err)
}
- log.Println("Listening at", listener.Addr())
+ log.Println("listening at", listener.Addr())
// Shut down the server gracefully (by closing the listener)
// if SIGTERM is received.
signal.Notify(term, syscall.SIGINT)
// Start serving requests.
- router = MakeRESTRouter(!cfg.DisableGet, !cfg.DisablePut, kc, time.Duration(cfg.Timeout), cfg.ManagementToken)
- http.Serve(listener, httpserver.AddRequestIDs(httpserver.LogRequests(router)))
-
- log.Println("shutting down")
+ router = MakeRESTRouter(kc, time.Duration(cluster.API.KeepServiceRequestTimeout), cluster.SystemRootToken)
+ return http.Serve(listener, httpserver.AddRequestIDs(httpserver.LogRequests(router)))
}
type ApiTokenCache struct {
// MakeRESTRouter returns an http.Handler that passes GET and PUT
// requests to the appropriate handlers.
-func MakeRESTRouter(enable_get bool, enable_put bool, kc *keepclient.KeepClient, timeout time.Duration, mgmtToken string) http.Handler {
+func MakeRESTRouter(kc *keepclient.KeepClient, timeout time.Duration, mgmtToken string) http.Handler {
rest := mux.NewRouter()
transport := defaultTransport
},
}
- if enable_get {
- rest.HandleFunc(`/{locator:[0-9a-f]{32}\+.*}`, h.Get).Methods("GET", "HEAD")
- rest.HandleFunc(`/{locator:[0-9a-f]{32}}`, h.Get).Methods("GET", "HEAD")
+ rest.HandleFunc(`/{locator:[0-9a-f]{32}\+.*}`, h.Get).Methods("GET", "HEAD")
+ rest.HandleFunc(`/{locator:[0-9a-f]{32}}`, h.Get).Methods("GET", "HEAD")
- // List all blocks
- rest.HandleFunc(`/index`, h.Index).Methods("GET")
+ // List all blocks
+ rest.HandleFunc(`/index`, h.Index).Methods("GET")
- // List blocks whose hash has the given prefix
- rest.HandleFunc(`/index/{prefix:[0-9a-f]{0,32}}`, h.Index).Methods("GET")
- }
+ // List blocks whose hash has the given prefix
+ rest.HandleFunc(`/index/{prefix:[0-9a-f]{0,32}}`, h.Index).Methods("GET")
- if enable_put {
- rest.HandleFunc(`/{locator:[0-9a-f]{32}\+.*}`, h.Put).Methods("PUT")
- rest.HandleFunc(`/{locator:[0-9a-f]{32}}`, h.Put).Methods("PUT")
- rest.HandleFunc(`/`, h.Put).Methods("POST")
- rest.HandleFunc(`/{any}`, h.Options).Methods("OPTIONS")
- rest.HandleFunc(`/`, h.Options).Methods("OPTIONS")
- }
+ rest.HandleFunc(`/{locator:[0-9a-f]{32}\+.*}`, h.Put).Methods("PUT")
+ rest.HandleFunc(`/{locator:[0-9a-f]{32}}`, h.Put).Methods("PUT")
+ rest.HandleFunc(`/`, h.Put).Methods("POST")
+ rest.HandleFunc(`/{any}`, h.Options).Methods("OPTIONS")
+ rest.HandleFunc(`/`, h.Options).Methods("OPTIONS")
rest.Handle("/_health/{check}", &health.Handler{
Token: mgmtToken,
Description=Arvados Keep Proxy
Documentation=https://doc.arvados.org/
After=network.target
-AssertPathExists=/etc/arvados/keepproxy/keepproxy.yml
# systemd==229 (ubuntu:xenial) obeys StartLimitInterval in the [Unit] section
StartLimitInterval=0
"math/rand"
"net/http"
"net/http/httptest"
- "os"
"strings"
"sync"
"testing"
"time"
+ "git.curoverse.com/arvados.git/lib/config"
+ "git.curoverse.com/arvados.git/sdk/go/arvados"
"git.curoverse.com/arvados.git/sdk/go/arvadosclient"
"git.curoverse.com/arvados.git/sdk/go/arvadostest"
"git.curoverse.com/arvados.git/sdk/go/keepclient"
+ log "github.com/sirupsen/logrus"
. "gopkg.in/check.v1"
)
arvadostest.StopAPI()
}
-func runProxy(c *C, args []string, bogusClientToken bool) *keepclient.KeepClient {
- args = append([]string{"keepproxy"}, args...)
- os.Args = append(args, "-listen=:0")
+func runProxy(c *C, bogusClientToken bool) *keepclient.KeepClient {
+ cfg, err := config.NewLoader(nil, nil).Load()
+ c.Assert(err, Equals, nil)
+ cluster, err := cfg.GetCluster("")
+ c.Assert(err, Equals, nil)
+
+ cluster.Services.Keepproxy.InternalURLs = map[arvados.URL]arvados.ServiceInstance{arvados.URL{Host: ":0"}: arvados.ServiceInstance{}}
+
listener = nil
- go main()
+ go func() {
+ run(log.New(), cluster)
+ defer closeListener()
+ }()
waitForListener()
- arv, err := arvadosclient.MakeArvadosClient()
+ client := arvados.NewClientFromEnv()
+ arv, err := arvadosclient.New(client)
c.Assert(err, Equals, nil)
if bogusClientToken {
arv.ApiToken = "bogus-token"
}
func (s *ServerRequiredSuite) TestResponseViaHeader(c *C) {
- runProxy(c, nil, false)
+ runProxy(c, false)
defer closeListener()
req, err := http.NewRequest("POST",
}
func (s *ServerRequiredSuite) TestLoopDetection(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
sr := map[string]string{
}
func (s *ServerRequiredSuite) TestStorageClassesHeader(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
// Set up fake keepstore to record request headers
}
func (s *ServerRequiredSuite) TestDesiredReplicas(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
content := []byte("TestDesiredReplicas")
}
func (s *ServerRequiredSuite) TestPutWrongContentLength(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
content := []byte("TestPutWrongContentLength")
// fixes the invalid Content-Length header. In order to test
// our server behavior, we have to call the handler directly
// using an httptest.ResponseRecorder.
- rtr := MakeRESTRouter(true, true, kc, 10*time.Second, "")
+ rtr := MakeRESTRouter(kc, 10*time.Second, "")
type testcase struct {
sendLength string
}
func (s *ServerRequiredSuite) TestManyFailedPuts(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
router.(*proxyHandler).timeout = time.Nanosecond
}
func (s *ServerRequiredSuite) TestPutAskGet(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
hash := fmt.Sprintf("%x", md5.Sum([]byte("foo")))
}
func (s *ServerRequiredSuite) TestPutAskGetForbidden(c *C) {
- kc := runProxy(c, nil, true)
+ kc := runProxy(c, true)
defer closeListener()
hash := fmt.Sprintf("%x+3", md5.Sum([]byte("bar")))
}
-func (s *ServerRequiredSuite) TestGetDisabled(c *C) {
- kc := runProxy(c, []string{"-no-get"}, false)
- defer closeListener()
-
- hash := fmt.Sprintf("%x", md5.Sum([]byte("baz")))
-
- {
- _, _, err := kc.Ask(hash)
- errNotFound, _ := err.(keepclient.ErrNotFound)
- c.Check(errNotFound, NotNil)
- c.Assert(err, ErrorMatches, `.*HTTP 405.*`)
- c.Log("Ask 1")
- }
-
- {
- hash2, rep, err := kc.PutB([]byte("baz"))
- c.Check(hash2, Matches, fmt.Sprintf(`^%s\+3(\+.+)?$`, hash))
- c.Check(rep, Equals, 2)
- c.Check(err, Equals, nil)
- c.Log("PutB")
- }
-
- {
- blocklen, _, err := kc.Ask(hash)
- errNotFound, _ := err.(keepclient.ErrNotFound)
- c.Check(errNotFound, NotNil)
- c.Assert(err, ErrorMatches, `.*HTTP 405.*`)
- c.Check(blocklen, Equals, int64(0))
- c.Log("Ask 2")
- }
-
- {
- _, blocklen, _, err := kc.Get(hash)
- errNotFound, _ := err.(keepclient.ErrNotFound)
- c.Check(errNotFound, NotNil)
- c.Assert(err, ErrorMatches, `.*HTTP 405.*`)
- c.Check(blocklen, Equals, int64(0))
- c.Log("Get")
- }
-}
-
-func (s *ServerRequiredSuite) TestPutDisabled(c *C) {
- kc := runProxy(c, []string{"-no-put"}, false)
- defer closeListener()
-
- hash2, rep, err := kc.PutB([]byte("quux"))
- c.Check(hash2, Equals, "")
- c.Check(rep, Equals, 0)
- c.Check(err, FitsTypeOf, keepclient.InsufficientReplicasError(errors.New("")))
-}
-
func (s *ServerRequiredSuite) TestCorsHeaders(c *C) {
- runProxy(c, nil, false)
+ runProxy(c, false)
defer closeListener()
{
}
func (s *ServerRequiredSuite) TestPostWithoutHash(c *C) {
- runProxy(c, nil, false)
+ runProxy(c, false)
defer closeListener()
{
// With a valid but non-existing prefix (expect "\n")
// With an invalid prefix (expect error)
func (s *ServerRequiredSuite) TestGetIndex(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
// Put "index-data" blocks
}
func (s *ServerRequiredSuite) TestCollectionSharingToken(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
hash, _, err := kc.PutB([]byte("shareddata"))
c.Check(err, IsNil)
}
func (s *ServerRequiredSuite) TestPutAskGetInvalidToken(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
// Put a test block
}
func (s *ServerRequiredSuite) TestAskGetKeepProxyConnectionError(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
// Point keepproxy at a non-existent keepstore
}
func (s *NoKeepServerSuite) TestAskGetNoKeepServerError(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
hash := fmt.Sprintf("%x", md5.Sum([]byte("foo")))
}
func (s *ServerRequiredSuite) TestPing(c *C) {
- kc := runProxy(c, nil, false)
+ kc := runProxy(c, false)
defer closeListener()
- rtr := MakeRESTRouter(true, true, kc, 10*time.Second, arvadostest.ManagementToken)
+ rtr := MakeRESTRouter(kc, 10*time.Second, arvadostest.ManagementToken)
req, err := http.NewRequest("GET",
"http://"+listener.Addr().String()+"/_health/ping",
+++ /dev/null
-// Copyright (C) The Arvados Authors. All rights reserved.
-//
-// SPDX-License-Identifier: AGPL-3.0
-
-package main
-
-import (
- "encoding/json"
- "flag"
- "fmt"
- "os"
-)
-
-func usage() {
- c := DefaultConfig()
- c.Client.APIHost = "zzzzz.arvadosapi.com:443"
- exampleConfigFile, err := json.MarshalIndent(c, " ", " ")
- if err != nil {
- panic(err)
- }
- fmt.Fprintf(os.Stderr, `
-
-Keepproxy forwards GET and PUT requests to keepstore servers. See
-http://doc.arvados.org/install/install-keepproxy.html
-
-Usage: keepproxy [-config path/to/keepproxy.yml]
-
-Options:
-`)
- flag.PrintDefaults()
- fmt.Fprintf(os.Stderr, `
-Example config file:
- %s
-
-Client.APIHost:
-
- Address (or address:port) of the Arvados API endpoint.
-
-Client.AuthToken:
-
- Anonymous API token.
-
-Client.Insecure:
-
- True if your Arvados API endpoint uses an unverifiable SSL/TLS
- certificate.
-
-Listen:
-
- Local port to listen on. Can be "address:port" or ":port", where
- "address" is a host IP address or name and "port" is a port number
- or name.
-
-DisableGet:
-
- Respond 404 to GET and HEAD requests.
-
-DisablePut:
-
- Respond 404 to PUT, POST, and OPTIONS requests.
-
-DefaultReplicas:
-
- Default number of replicas to write if not specified by the
- client. If this is zero or omitted, the site-wide
- defaultCollectionReplication configuration will be used.
-
-Timeout:
-
- Timeout for requests to keep services, with units (e.g., "120s",
- "2m").
-
-PIDFile:
-
- Path to PID file. During startup this file will be created if
- needed, and locked using flock() until keepproxy exits. If it is
- already locked, or any error is encountered while writing to it,
- keepproxy will exit immediately. If omitted or empty, no PID file
- will be used.
-
-Debug:
-
- Enable debug logging.
-
-ManagementToken:
-
- Authorization token to be included in all health check requests.
-
-`, exampleConfigFile)
-}
ExternalURL: "https://$localip:${services[workbench2-ssl]}"
SSO:
ExternalURL: "https://$localip:${services[sso]}"
+ Keepproxy:
+ InternalURLs:
+ "http://localhost:${services[keepproxy]}/": {}
+ ExternalURL: "http://$localip:${services[keepproxy-ssl]}/"
Websocket:
ExternalURL: "wss://$localip:${services[websockets-ssl]}/websocket"
InternalURLs:
echo $UUID > /var/lib/arvados/keepproxy-uuid
fi
-exec /usr/local/bin/keepproxy -listen=:${services[keepproxy]}
+exec /usr/local/bin/keepproxy
projects / arvados.git / commitdiff