Documentation
¶
Index ¶
- Variables
- func ChooseInstanceType(cc *arvados.Cluster, ctr *arvados.Container) ([]arvados.InstanceType, error)
- func EstimateScratchSpace(ctr *arvados.Container) (needScratch int64)
- type APIClient
- type ConstraintsNotSatisfiableError
- type InstanceResources
- type Queue
- func (cq *Queue) Cancel(uuid string) error
- func (cq *Queue) Entries() (entries map[string]QueueEnt, threshold time.Time)
- func (cq *Queue) Forget(uuid string)
- func (cq *Queue) Get(uuid string) (arvados.Container, bool)
- func (cq *Queue) Lock(uuid string) error
- func (cq *Queue) Subscribe() <-chan struct{}
- func (cq *Queue) Unlock(uuid string) error
- func (cq *Queue) Unsubscribe(ch <-chan struct{})
- func (cq *Queue) Update() error
- type QueueEnt
Constants ¶
This section is empty.
Variables ¶
Functions ¶
func ChooseInstanceType ¶
func ChooseInstanceType(cc *arvados.Cluster, ctr *arvados.Container) ([]arvados.InstanceType, error)
ChooseInstanceType returns the arvados.InstanceTypes eligible to run ctr, i.e., those that have enough RAM, VCPUs, etc., and are not too expensive according to cluster configuration.
The returned types are sorted with lower prices first.
The error is non-nil if and only if the returned slice is empty.
func EstimateScratchSpace ¶
EstimateScratchSpace estimates how much available disk space (in bytes) is needed to run the container by summing the capacity requested by 'tmp' mounts plus disk space required to load the Docker image plus arv-mount block cache.
Types ¶
type APIClient ¶
type APIClient interface {
RequestAndDecode(dst interface{}, method, path string, body io.Reader, params interface{}) error
}
An APIClient performs Arvados API requests. It is typically an *arvados.Client.
type ConstraintsNotSatisfiableError ¶
type ConstraintsNotSatisfiableError struct {
AvailableTypes []arvados.InstanceType
// contains filtered or unexported fields
}
ConstraintsNotSatisfiableError includes a list of available instance types to be reported back to the user.
type InstanceResources ¶
type InstanceResources struct {
VCPUs int
RAM arvados.ByteSize
Scratch arvados.ByteSize
GPUs int
GPUVRAM arvados.ByteSize
// contains filtered or unexported fields
}
InstanceResources are the allocatable resources needed to run a container, including system overhead.
func InstanceResourcesNeeded ¶
func InstanceResourcesNeeded(cc *arvados.Cluster, ctr *arvados.Container) InstanceResources
func (InstanceResources) Accommodates ¶
func (r InstanceResources) Accommodates(r2 InstanceResources) bool
Check whether an instance's remaining resources r are enough to accommodate container r2.
If an instance is running any 0-VCPU containers, only r.VCPUs-1 are available for containers that request 1 or more VCPUs.
func (InstanceResources) Sub ¶
func (r InstanceResources) Sub(r2 InstanceResources) InstanceResources
Subtract r2 from the resources r remaining on an instance.
type Queue ¶
type Queue struct {
// contains filtered or unexported fields
}
A Queue is an interface to an Arvados cluster's container database. It presents only the containers that are eligible to be run by, are already being run by, or have recently been run by the present dispatcher.
The Entries, Get, and Forget methods do not block: they return immediately, using cached data.
The updating methods (Cancel, Lock, Unlock, Update) do block: they return only after the operation has completed.
A Queue's Update method should be called periodically to keep the cache up to date.
func NewQueue ¶
func NewQueue(logger logrus.FieldLogger, reg *prometheus.Registry, cluster *arvados.Cluster, client APIClient) *Queue
NewQueue returns a new Queue. When a new container appears in the Arvados cluster's queue during Update, chooseType will be called to assign an appropriate arvados.InstanceType for the queue entry.
func (*Queue) Cancel ¶
Cancel cancels the given container.
func (*Queue) Entries ¶
Entries returns all cache entries, keyed by container UUID.
The returned threshold indicates the maximum age of any cached data returned in the map. This makes it possible for a scheduler to determine correctly the outcome of a remote process that updates container state. It must first wait for the remote process to exit, then wait for the Queue to start and finish its next Update -- i.e., it must wait until threshold > timeProcessExited.
func (*Queue) Forget ¶
Forget drops the specified container from the cache. It should be called on finalized containers to avoid leaking memory over time. It is a no-op if the indicated container is not in a finalized state.
func (*Queue) Get ¶
Get returns the (partial) Container record for the specified container. Like a map lookup, its second return value is false if the specified container is not in the Queue.
func (*Queue) Lock ¶
Lock acquires the dispatch lock for the given container.
func (*Queue) Subscribe ¶
func (cq *Queue) Subscribe() <-chan struct{}
Subscribe returns a channel that becomes ready to receive when an entry in the Queue is updated.
ch := q.Subscribe()
defer q.Unsubscribe(ch)
for range ch {
// ...
}
func (*Queue) Unlock ¶
Unlock releases the dispatch lock for the given container.
type QueueEnt ¶
type QueueEnt struct {
// The container to run. Only the UUID, State, Priority,
// RuntimeConstraints, ContainerImage, SchedulingParameters,
// and CreatedAt fields are populated.
Container arvados.Container `json:"container"`
InstanceResources InstanceResources `json:"instance_resources"`
InstanceTypes []arvados.InstanceType `json:"instance_types"`
FirstSeenAt time.Time `json:"first_seen_at"`
}
A QueueEnt is an entry in the queue, consisting of a container record and the instance type that should be used to run it.
Source Files
¶
- node_size.go
- queue.go