Storage classes (alternately known as "storage tiers") allow you to control which volumes should be used to store particular collection data blocks. This can be used to implement data storage policies such as moving data to archival storage.
-The storage classes for each volume are set in the per-volume "keepstore configuration":{{site.baseurl}}/install/install-keepstore.html
+In the default Arvados configuration, with no storage classes specified in the configuration file, all volumes belong to a single implicit storage class called "default". Apart from that, names of storage classes are internal to the cluster and decided by the administrator. Other than the implicit "default" class, Arvados currently does not define any standard storage class names.
+
+To use multiple storage classes, update the @StorageClasses@ and @Volumes@ sections of your configuration file.
+* Every storage class you use (including "default") must be defined in the @StorageClasses@ section.
+* The @StorageClasses@ section must use @Default: true@ to indicate at least one default storage class. When a client/user does not specify storage classes when creating a new collection, the default storage classes are used implicitly.
+* If some storage classes are faster or cheaper to access than others, assign a higher @Priority@ to the faster ones. When reading data, volumes with high priority storage classes are searched first.
+
+Example:
<pre>
+ StorageClasses:
+
+ default:
+ # When reading a block that is stored on multiple volumes,
+ # prefer a volume with this class.
+ Priority: 20
+
+ # When a client does not specify a storage class when saving a
+ # new collection, use this one.
+ Default: true
+
+ archival:
+ Priority: 10
+
Volumes:
+
ClusterID-nyw5e-000000000000000:
# This volume is in the "default" storage class.
StorageClasses:
default: true
+
ClusterID-nyw5e-000000000000001:
- # Specify this volume is in the "archival" storage class.
+ # This volume is in the "archival" storage class.
StorageClasses:
archival: true
</pre>
-Names of storage classes are internal to the cluster and decided by the administrator. Aside from "default", Arvados currently does not define any standard storage class names.
+Refer to the "configuration reference":{{site.baseurl}}/admin/config.html for more details.
h3. Using storage classes
"Upgrading from 2.2.0":#v2_2_0
+h3. Storage classes must be defined explicitly
+
+If your configuration uses the StorageClasses attribute on any Keep volumes, you must add a new @StorageClasses@ section that lists all of your storage classes. Refer to the updated documentation about "configuring storage classes":{{site.baseurl}}/admin/storage-classes.html for details.
+
h3. keep-balance requires access to PostgreSQL
Make sure the keep-balance process can connect to your PostgreSQL server using the settings in your config file. (In previous versions, keep-balance accessed the database through controller instead of connecting to the database server directly.)
$ arvados-cwl-runner --storage-classes=hot myworkflow.cwl myinput.yml
</pre>
-(Note: intermediate collections produced by a workflow run will have "default" storage class.)
+(Note: intermediate collections produced by a workflow run will use the cluster's default storage class(es).)
h3. arv command line
h3. Storage class notes
-Collection blocks will be in the "default" storage class if not otherwise specified.
+Collection blocks will be in the cluster's configured default storage class(es) if not otherwise specified.
Any user with write access to a collection may set any storage class on that collection.
-
-Names of storage classes are internal to the cluster and decided by the administrator. Aside from "default", Arvados currently does not define any standard storage class names.
},
}
}
+ cluster.StorageClasses = map[string]arvados.StorageClassConfig{
+ "default": {Default: true},
+ "foo": {},
+ "bar": {},
+ }
}
if super.OwnTemporaryDatabase {
cluster.PostgreSQL.Connection = arvados.PostgreSQLConnection{
Price: 0.1
Preemptible: false
+ StorageClasses:
+
+ # If you use multiple storage classes, specify them here, using
+ # the storage class name as the key (in place of "SAMPLE" in
+ # this sample entry).
+ #
+ # Further info/examples:
+ # https://doc.arvados.org/admin/storage-classes.html
+ SAMPLE:
+
+ # Priority determines the order volumes should be searched
+ # when reading data, in cases where a keepstore server has
+ # access to multiple volumes with different storage classes.
+ Priority: 0
+
+ # Default determines which storage class(es) should be used
+ # when a user/client writes data or saves a new collection
+ # without specifying storage classes.
+ #
+ # If any StorageClasses are configured, at least one of them
+ # must have Default: true.
+ Default: true
+
Volumes:
SAMPLE:
# AccessViaHosts specifies which keepstore processes can read
ReadOnly: false
Replication: 1
StorageClasses:
- default: true
+ # If you have configured storage classes (see StorageClasses
+ # section above), add an entry here for each storage class
+ # satisfied by this volume.
SAMPLE: true
Driver: S3
DriverParameters:
"Services.*": true,
"Services.*.ExternalURL": true,
"Services.*.InternalURLs": false,
+ "StorageClasses": true,
+ "StorageClasses.*": true,
+ "StorageClasses.*.Default": true,
+ "StorageClasses.*.Priority": true,
"SystemLogs": false,
"SystemRootToken": false,
"TLS": false,
Price: 0.1
Preemptible: false
+ StorageClasses:
+
+ # If you use multiple storage classes, specify them here, using
+ # the storage class name as the key (in place of "SAMPLE" in
+ # this sample entry).
+ #
+ # Further info/examples:
+ # https://doc.arvados.org/admin/storage-classes.html
+ SAMPLE:
+
+ # Priority determines the order volumes should be searched
+ # when reading data, in cases where a keepstore server has
+ # access to multiple volumes with different storage classes.
+ Priority: 0
+
+ # Default determines which storage class(es) should be used
+ # when a user/client writes data or saves a new collection
+ # without specifying storage classes.
+ #
+ # If any StorageClasses are configured, at least one of them
+ # must have Default: true.
+ Default: true
+
Volumes:
SAMPLE:
# AccessViaHosts specifies which keepstore processes can read
ReadOnly: false
Replication: 1
StorageClasses:
- default: true
+ # If you have configured storage classes (see StorageClasses
+ # section above), add an entry here for each storage class
+ # satisfied by this volume.
SAMPLE: true
Driver: S3
DriverParameters:
ldr.loadOldKeepBalanceConfig,
)
}
+ loadFuncs = append(loadFuncs, ldr.setImplicitStorageClasses)
for _, f := range loadFuncs {
err = f(&cfg)
if err != nil {
checkKeyConflict(fmt.Sprintf("Clusters.%s.PostgreSQL.Connection", id), cc.PostgreSQL.Connection),
ldr.checkEmptyKeepstores(cc),
ldr.checkUnlistedKeepstores(cc),
+ ldr.checkStorageClasses(cc),
// TODO: check non-empty Rendezvous on
// services other than Keepstore
} {
return nil
}
+func (ldr *Loader) setImplicitStorageClasses(cfg *arvados.Config) error {
+cluster:
+ for id, cc := range cfg.Clusters {
+ if len(cc.StorageClasses) > 0 {
+ continue cluster
+ }
+ for _, vol := range cc.Volumes {
+ if len(vol.StorageClasses) > 0 {
+ continue cluster
+ }
+ }
+ // No explicit StorageClasses config info at all; fill
+ // in implicit defaults.
+ for id, vol := range cc.Volumes {
+ vol.StorageClasses = map[string]bool{"default": true}
+ cc.Volumes[id] = vol
+ }
+ cc.StorageClasses = map[string]arvados.StorageClassConfig{"default": {Default: true}}
+ cfg.Clusters[id] = cc
+ }
+ return nil
+}
+
+func (ldr *Loader) checkStorageClasses(cc arvados.Cluster) error {
+ classOnVolume := map[string]bool{}
+ for volid, vol := range cc.Volumes {
+ if len(vol.StorageClasses) == 0 {
+ return fmt.Errorf("%s: volume has no StorageClasses listed", volid)
+ }
+ for classid := range vol.StorageClasses {
+ if _, ok := cc.StorageClasses[classid]; !ok {
+ return fmt.Errorf("%s: volume refers to storage class %q that is not defined in StorageClasses", volid, classid)
+ }
+ classOnVolume[classid] = true
+ }
+ }
+ haveDefault := false
+ for classid, sc := range cc.StorageClasses {
+ if !classOnVolume[classid] && len(cc.Volumes) > 0 {
+ ldr.Logger.Warnf("there are no volumes providing storage class %q", classid)
+ }
+ if sc.Default {
+ haveDefault = true
+ }
+ }
+ if !haveDefault {
+ return fmt.Errorf("there is no default storage class (at least one entry in StorageClasses must have Default: true)")
+ }
+ return nil
+}
+
func checkKeyConflict(label string, m map[string]string) error {
saw := map[string]bool{}
for k := range m {
var _ = check.Suite(&LoadSuite{})
+var emptyConfigYAML = `Clusters: {"z1111": {}}`
+
// Return a new Loader that reads cluster config from configdata
// (instead of the usual default /etc/arvados/config.yml), and logs to
// logdst or (if that's nil) c.Log.
}
func (s *LoadSuite) TestNoConfigs(c *check.C) {
- cfg, err := testLoader(c, `Clusters: {"z1111": {}}`, nil).Load()
+ cfg, err := testLoader(c, emptyConfigYAML, nil).Load()
c.Assert(err, check.IsNil)
c.Assert(cfg.Clusters, check.HasLen, 1)
cc, err := cfg.GetCluster("z1111")
f, err = ioutil.TempFile("", "")
c.Check(err, check.IsNil)
defer os.Remove(f.Name())
- io.WriteString(f, "Clusters: {aaaaa: {}}\n")
+ io.WriteString(f, emptyConfigYAML)
newfile := f.Name()
for _, trial := range []struct {
c.Errorf("Should have produced an error")
}
- var logbuf bytes.Buffer
- loader := testLoader(c, string(DefaultYAML), &logbuf)
+ loader := testLoader(c, string(DefaultYAML), nil)
cfg, err := loader.Load()
c.Assert(err, check.IsNil)
if err := checkListKeys("", cfg); err != nil {
c.Error(err)
}
}
+
+func (s *LoadSuite) TestImplicitStorageClasses(c *check.C) {
+ // If StorageClasses and Volumes.*.StorageClasses are all
+ // empty, there is a default storage class named "default".
+ ldr := testLoader(c, `{"Clusters":{"z1111":{}}}`, nil)
+ cfg, err := ldr.Load()
+ c.Assert(err, check.IsNil)
+ cc, err := cfg.GetCluster("z1111")
+ c.Assert(err, check.IsNil)
+ c.Check(cc.StorageClasses, check.HasLen, 1)
+ c.Check(cc.StorageClasses["default"].Default, check.Equals, true)
+ c.Check(cc.StorageClasses["default"].Priority, check.Equals, 0)
+
+ // The implicit "default" storage class is used by all
+ // volumes.
+ ldr = testLoader(c, `
+Clusters:
+ z1111:
+ Volumes:
+ z: {}`, nil)
+ cfg, err = ldr.Load()
+ c.Assert(err, check.IsNil)
+ cc, err = cfg.GetCluster("z1111")
+ c.Assert(err, check.IsNil)
+ c.Check(cc.StorageClasses, check.HasLen, 1)
+ c.Check(cc.StorageClasses["default"].Default, check.Equals, true)
+ c.Check(cc.StorageClasses["default"].Priority, check.Equals, 0)
+ c.Check(cc.Volumes["z"].StorageClasses["default"], check.Equals, true)
+
+ // The "default" storage class isn't implicit if any classes
+ // are configured explicitly.
+ ldr = testLoader(c, `
+Clusters:
+ z1111:
+ StorageClasses:
+ local:
+ Default: true
+ Priority: 111
+ Volumes:
+ z:
+ StorageClasses:
+ local: true`, nil)
+ cfg, err = ldr.Load()
+ c.Assert(err, check.IsNil)
+ cc, err = cfg.GetCluster("z1111")
+ c.Assert(err, check.IsNil)
+ c.Check(cc.StorageClasses, check.HasLen, 1)
+ c.Check(cc.StorageClasses["local"].Default, check.Equals, true)
+ c.Check(cc.StorageClasses["local"].Priority, check.Equals, 111)
+
+ // It is an error for a volume to refer to a storage class
+ // that isn't listed in StorageClasses.
+ ldr = testLoader(c, `
+Clusters:
+ z1111:
+ StorageClasses:
+ local:
+ Default: true
+ Priority: 111
+ Volumes:
+ z:
+ StorageClasses:
+ nx: true`, nil)
+ _, err = ldr.Load()
+ c.Assert(err, check.ErrorMatches, `z: volume refers to storage class "nx" that is not defined.*`)
+
+ // It is an error for a volume to refer to a storage class
+ // that isn't listed in StorageClasses ... even if it's
+ // "default", which would exist implicitly if it weren't
+ // referenced explicitly by a volume.
+ ldr = testLoader(c, `
+Clusters:
+ z1111:
+ Volumes:
+ z:
+ StorageClasses:
+ default: true`, nil)
+ _, err = ldr.Load()
+ c.Assert(err, check.ErrorMatches, `z: volume refers to storage class "default" that is not defined.*`)
+
+ // If the "default" storage class is configured explicitly, it
+ // is not used implicitly by any volumes, even if it's the
+ // only storage class.
+ var logbuf bytes.Buffer
+ ldr = testLoader(c, `
+Clusters:
+ z1111:
+ StorageClasses:
+ default:
+ Default: true
+ Priority: 111
+ Volumes:
+ z: {}`, &logbuf)
+ _, err = ldr.Load()
+ c.Assert(err, check.ErrorMatches, `z: volume has no StorageClasses listed`)
+
+ // If StorageClasses are configured explicitly, there must be
+ // at least one with Default: true. (Calling one "default" is
+ // not sufficient.)
+ ldr = testLoader(c, `
+Clusters:
+ z1111:
+ StorageClasses:
+ default:
+ Priority: 111
+ Volumes:
+ z:
+ StorageClasses:
+ default: true`, nil)
+ _, err = ldr.Load()
+ c.Assert(err, check.ErrorMatches, `there is no default storage class.*`)
+}
PreferDomainForUsername string
UserSetupMailText string
}
- Volumes map[string]Volume
- Workbench struct {
+ StorageClasses map[string]StorageClassConfig
+ Volumes map[string]Volume
+ Workbench struct {
ActivationContactLink string
APIClientConnectTimeout Duration
APIClientReceiveTimeout Duration
}
}
+type StorageClassConfig struct {
+ Default bool
+ Priority int
+}
+
type Volume struct {
AccessViaHosts map[URL]VolumeAccess
ReadOnly bool
self._local_collection = arvados.collection.Collection(
self._state['manifest'],
replication_desired=self.replication_desired,
- storage_classes_desired=(self.storage_classes or ['default']),
+ storage_classes_desired=self.storage_classes,
put_threads=self.put_threads,
api_client=self._api_client,
num_retries=self.num_retries)
# Posgresql JSONB columns should NOT be declared as serialized, Rails 5
# already know how to properly treat them.
attribute :properties, :jsonbHash, default: {}
- attribute :storage_classes_desired, :jsonbArray, default: ["default"]
+ attribute :storage_classes_desired, :jsonbArray, default: lambda { Rails.configuration.DefaultStorageClasses }
attribute :storage_classes_confirmed, :jsonbArray, default: []
before_validation :default_empty_manifest
# validation on empty desired storage classes return an error.
def default_storage_classes
if self.storage_classes_desired.nil? || self.storage_classes_desired.empty?
- self.storage_classes_desired = ["default"]
+ self.storage_classes_desired = Rails.configuration.DefaultStorageClasses
end
self.storage_classes_confirmed ||= []
end
attribute :secret_mounts, :jsonbHash, default: {}
attribute :runtime_status, :jsonbHash, default: {}
attribute :runtime_auth_scopes, :jsonbArray, default: []
- attribute :output_storage_classes, :jsonbArray, default: ["default"]
+ attribute :output_storage_classes, :jsonbArray, default: lambda { Rails.configuration.DefaultStorageClasses }
serialize :environment, Hash
serialize :mounts, Hash
# already know how to properly treat them.
attribute :properties, :jsonbHash, default: {}
attribute :secret_mounts, :jsonbHash, default: {}
- attribute :output_storage_classes, :jsonbArray, default: ["default"]
+ attribute :output_storage_classes, :jsonbArray, default: lambda { Rails.configuration.DefaultStorageClasses }
serialize :environment, Hash
serialize :mounts, Hash
ConfigLoader.set_cfg cfg, "RemoteClusters", h
}
arvcfg.declare_config "RemoteClusters.*.Proxy", Boolean, :remote_hosts_via_dns
+arvcfg.declare_config "StorageClasses", Hash
dbcfg = ConfigLoader.new
raise "default_trash_lifetime is %d, must be at least 86400" % Rails.configuration.Collections.DefaultTrashLifetime
end
+default_storage_classes = []
+$arvados_config["StorageClasses"].each do |cls, cfg|
+ if cfg["Default"]
+ default_storage_classes << cls
+ end
+end
+if default_storage_classes.length == 0
+ default_storage_classes = ["default"]
+end
+$arvados_config["DefaultStorageClasses"] = default_storage_classes.sort
+
#
# Special case for test database where there's no database.yml,
# because the Arvados config.yml doesn't have a concept of multiple
end
end
+ test "storage_classes_desired default respects config" do
+ saved = Rails.configuration.DefaultStorageClasses
+ Rails.configuration.DefaultStorageClasses = ["foo"]
+ begin
+ act_as_user users(:active) do
+ c = Collection.create!
+ assert_equal ["foo"], c.storage_classes_desired
+ end
+ ensure
+ Rails.configuration.DefaultStorageClasses = saved
+ end
+ end
+
test "storage_classes_desired cannot be empty" do
act_as_user users(:active) do
c = collections(:collection_owned_by_active)
end
test "default output_storage_classes" do
- act_as_user users(:active) do
- cr = create_minimal_req!(priority: 1,
- state: ContainerRequest::Committed,
- output_name: 'foo')
- run_container(cr)
- cr.reload
- output = Collection.find_by_uuid(cr.output_uuid)
- assert_equal ["default"], output.storage_classes_desired
+ saved = Rails.configuration.DefaultStorageClasses
+ Rails.configuration.DefaultStorageClasses = ["foo"]
+ begin
+ act_as_user users(:active) do
+ cr = create_minimal_req!(priority: 1,
+ state: ContainerRequest::Committed,
+ output_name: 'foo')
+ run_container(cr)
+ cr.reload
+ output = Collection.find_by_uuid(cr.output_uuid)
+ assert_equal ["foo"], output.storage_classes_desired
+ end
+ ensure
+ Rails.configuration.DefaultStorageClasses = saved
end
end
// effectively read-only.
mnt.ReadOnly = mnt.ReadOnly || srv.ReadOnly
- if len(mnt.StorageClasses) == 0 {
- bal.mountsByClass["default"][mnt] = true
- continue
- }
for class := range mnt.StorageClasses {
if mbc := bal.mountsByClass[class]; mbc == nil {
bal.classes = append(bal.classes, class)
var stubMounts = map[string][]arvados.KeepMount{
"keep0.zzzzz.arvadosapi.com:25107": {{
- UUID: "zzzzz-ivpuk-000000000000000",
- DeviceID: "keep0-vol0",
+ UUID: "zzzzz-ivpuk-000000000000000",
+ DeviceID: "keep0-vol0",
+ StorageClasses: map[string]bool{"default": true},
}},
"keep1.zzzzz.arvadosapi.com:25107": {{
- UUID: "zzzzz-ivpuk-100000000000000",
- DeviceID: "keep1-vol0",
+ UUID: "zzzzz-ivpuk-100000000000000",
+ DeviceID: "keep1-vol0",
+ StorageClasses: map[string]bool{"default": true},
}},
"keep2.zzzzz.arvadosapi.com:25107": {{
- UUID: "zzzzz-ivpuk-200000000000000",
- DeviceID: "keep2-vol0",
+ UUID: "zzzzz-ivpuk-200000000000000",
+ DeviceID: "keep2-vol0",
+ StorageClasses: map[string]bool{"default": true},
}},
"keep3.zzzzz.arvadosapi.com:25107": {{
- UUID: "zzzzz-ivpuk-300000000000000",
- DeviceID: "keep3-vol0",
+ UUID: "zzzzz-ivpuk-300000000000000",
+ DeviceID: "keep3-vol0",
+ StorageClasses: map[string]bool{"default": true},
}},
}
}
srv.mounts = []*KeepMount{{
KeepMount: arvados.KeepMount{
- UUID: fmt.Sprintf("zzzzz-mount-%015x", i),
+ UUID: fmt.Sprintf("zzzzz-mount-%015x", i),
+ StorageClasses: map[string]bool{"default": true},
},
KeepService: srv,
}}
srv.mounts[0].KeepMount.DeviceID = fmt.Sprintf("writable-by-srv-%x", i)
srv.mounts = append(srv.mounts, &KeepMount{
KeepMount: arvados.KeepMount{
- DeviceID: fmt.Sprintf("writable-by-srv-%x", (i+1)%len(bal.srvs)),
- UUID: fmt.Sprintf("zzzzz-mount-%015x", i<<16),
- ReadOnly: readonly,
- Replication: 1,
+ DeviceID: fmt.Sprintf("writable-by-srv-%x", (i+1)%len(bal.srvs)),
+ UUID: fmt.Sprintf("zzzzz-mount-%015x", i<<16),
+ ReadOnly: readonly,
+ Replication: 1,
+ StorageClasses: map[string]bool{"default": true},
},
KeepService: srv,
})
"net/http"
"net/http/httptest"
"os"
- "regexp"
"sort"
"strings"
"time"
}
}
+func (s *HandlerSuite) TestReadsOrderedByStorageClassPriority(c *check.C) {
+ s.cluster.Volumes = map[string]arvados.Volume{
+ "zzzzz-nyw5e-111111111111111": {
+ Driver: "mock",
+ Replication: 1,
+ StorageClasses: map[string]bool{"class1": true}},
+ "zzzzz-nyw5e-222222222222222": {
+ Driver: "mock",
+ Replication: 1,
+ StorageClasses: map[string]bool{"class2": true, "class3": true}},
+ }
+
+ for _, trial := range []struct {
+ priority1 int // priority of class1, thus vol1
+ priority2 int // priority of class2
+ priority3 int // priority of class3 (vol2 priority will be max(priority2, priority3))
+ get1 int // expected number of "get" ops on vol1
+ get2 int // expected number of "get" ops on vol2
+ }{
+ {100, 50, 50, 1, 0}, // class1 has higher priority => try vol1 first, no need to try vol2
+ {100, 100, 100, 1, 0}, // same priority, vol1 is first lexicographically => try vol1 first and succeed
+ {66, 99, 33, 1, 1}, // class2 has higher priority => try vol2 first, then try vol1
+ {66, 33, 99, 1, 1}, // class3 has highest priority => vol2 has highest => try vol2 first, then try vol1
+ } {
+ c.Logf("%+v", trial)
+ s.cluster.StorageClasses = map[string]arvados.StorageClassConfig{
+ "class1": {Priority: trial.priority1},
+ "class2": {Priority: trial.priority2},
+ "class3": {Priority: trial.priority3},
+ }
+ c.Assert(s.handler.setup(context.Background(), s.cluster, "", prometheus.NewRegistry(), testServiceURL), check.IsNil)
+ IssueRequest(s.handler,
+ &RequestTester{
+ method: "PUT",
+ uri: "/" + TestHash,
+ requestBody: TestBlock,
+ storageClasses: "class1",
+ })
+ IssueRequest(s.handler,
+ &RequestTester{
+ method: "GET",
+ uri: "/" + TestHash,
+ })
+ c.Check(s.handler.volmgr.mountMap["zzzzz-nyw5e-111111111111111"].Volume.(*MockVolume).CallCount("Get"), check.Equals, trial.get1)
+ c.Check(s.handler.volmgr.mountMap["zzzzz-nyw5e-222222222222222"].Volume.(*MockVolume).CallCount("Get"), check.Equals, trial.get2)
+ }
+}
+
// Test TOUCH requests.
func (s *HandlerSuite) TestTouchHandler(c *check.C) {
c.Assert(s.handler.setup(context.Background(), s.cluster, "", prometheus.NewRegistry(), testServiceURL), check.IsNil)
expected := `^` + TestHash + `\+\d+ \d+\n` +
TestHash2 + `\+\d+ \d+\n\n$`
- match, _ := regexp.MatchString(expected, response.Body.String())
- if !match {
- c.Errorf(
- "permissions on, superuser request: expected %s, got:\n%s",
- expected, response.Body.String())
- }
+ c.Check(response.Body.String(), check.Matches, expected, check.Commentf(
+ "permissions on, superuser request"))
// superuser /index/prefix request
// => OK
response)
expected = `^` + TestHash + `\+\d+ \d+\n\n$`
- match, _ = regexp.MatchString(expected, response.Body.String())
- if !match {
- c.Errorf(
- "permissions on, superuser /index/prefix request: expected %s, got:\n%s",
- expected, response.Body.String())
- }
+ c.Check(response.Body.String(), check.Matches, expected, check.Commentf(
+ "permissions on, superuser /index/prefix request"))
// superuser /index/{no-such-prefix} request
// => OK
for i, sc := range wantStorageClasses {
wantStorageClasses[i] = strings.TrimSpace(sc)
}
+ } else {
+ // none specified -- use configured default
+ for class, cfg := range rtr.cluster.StorageClasses {
+ if cfg.Default {
+ wantStorageClasses = append(wantStorageClasses, class)
+ }
+ }
}
buf, err := getBufferWithContext(ctx, bufs, int(req.ContentLength))
"fmt"
"io"
"math/big"
+ "sort"
"sync/atomic"
"time"
vm.writables = append(vm.writables, mnt)
}
}
+ // pri(i): return highest priority of any storage class
+ // offered by vm.readables[i]
+ pri := func(i int) int {
+ any, best := false, 0
+ for class := range vm.readables[i].KeepMount.StorageClasses {
+ if p := cluster.StorageClasses[class].Priority; !any || best < p {
+ best = p
+ any = true
+ }
+ }
+ return best
+ }
+ // sort vm.readables, first by highest priority of any offered
+ // storage class (highest->lowest), then by volume UUID
+ sort.Slice(vm.readables, func(i, j int) bool {
+ if pi, pj := pri(i), pri(j); pi != pj {
+ return pi > pj
+ } else {
+ return vm.readables[i].KeepMount.UUID < vm.readables[j].KeepMount.UUID
+ }
+ })
return vm, nil
}
projects / arvados.git / commitdiff