Kubernetes Borg/Omega history topic 5: Asynchronous controllers. Borgmaster had synchronous, transactional, edge-triggered state machines. We had challenges scaling, evolving, and extending them.
-
Show this thread
-
High-cardinality resource instances could exceed what could be done in a single transaction. Addition of new states broke clients. Unobserved changes could cause unexpected state transitions. Adding new resource types was hard, and would have had to be added to monolithic files.
1 reply 0 retweets 3 likesShow this thread -
As a result, when new teams worked on new functionality, such as batch scheduling and autoscaling, they built it into external components, which were asynchronous. Ingestion of status from nodes (Borglets) was asynchronous, as well. Omega embraced asynchronous controllers
1 reply 0 retweets 2 likesShow this thread -
Omega represented desired state and observed state in separate records in its transactional Paxos store. This made it harder to assemble a picture of what was going on. In Kubernetes, we decided to represent status in the same object as spec, in v1beta3:http://issues.k8s.io/1225
3 replies 0 retweets 5 likesShow this thread -
Replying to @bgrant0607
It's represented as the same object, but Status is a subresource (with its own storage in etcd) is it not?
1 reply 0 retweets 0 likes
It is a subresource, to prevent spec and status changes from accidentally stepping on each other and to enable distinct ACLs, but they don't have separate storage yet. Updating 2 resources in etcd2 (e.g. BoundPod) was painful.http://issues.k8s.io/8625
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.