Kubernetes Borg/Omega history topic 9: Scheduling constraints. I have volumes more to write about configuration, but will move on with history topics for now. Borg's set of constraints grew organically over time. It started with just required memory, before multicore and NPTL
-
Show this thread
-
Other resources were added: cpu, disks. Hard and soft constraints on key/value machine attributes, and “attribute limits” to limit the number of tasks per failure domain. Automatically injected anti-constraints were used to implement dedicated machines
1 reply 0 retweets 0 likesShow this thread -
In Omega (https://ai.google/research/pubs/pub41684 …), we added the concepts of taints and tolerations in order to subsume a number of ad hoc means to prevent scheduling of most tasks and/or evict them from certain machines, and forgiveness to defer eviction
1 reply 1 retweet 3 likesShow this thread -
These scheduling features made their way pretty directly into Kubernetes: http://issues.k8s.io/168 , http://issues.k8s.io/367 , http://issues.k8s.io/1574 , http://issues.k8s.io/17190 .
@davidopp, who was the TL of scheduling in Borg and Omega, worked on a number of these features in K8s too1 reply 0 retweets 3 likesShow this thread -
A scheduling braindump I wrote in early 2015 (https://github.com/kubernetes/kubernetes/issues/4301#issuecomment-74355529 …) possibly helped to convince some that Google really was fully sharing its experience with the project. The scheduling design docs can be found in https://github.com/kubernetes/community/tree/master/contributors/design-proposals/scheduling …
1 reply 0 retweets 5 likesShow this thread -
These mechanisms can be used to manage how workloads are binpacked for efficiency, spread for availability, isolated from one another for performance or reliability or security, colocated with required resources, matched with desired configurations, and manage node drains
1 reply 0 retweets 4 likesShow this thread -
These scheduling primitives are pretty flexible, but if there are constraints or other policies or criteria that can’t be represented, users can use their own schedulers. In order to do that in Borg, one would have to add a constraint to a task to pin it to a specific machine
1 reply 0 retweets 2 likesShow this thread -
The Omega paper compared performance of 2-level scheduling with information hiding, but one issue it didn’t mention is that the lower-level scheduler needs to implement all of the same constraints as all the upper-level schedulers, or it may never satisfy their requirements
1 reply 1 retweet 6 likesShow this thread
Anyway, while resource optimization is an important concern, there are many other considerations in decisions, such as whether container images already resident, which facilitates faster start time
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.