Kubernetes Borg/Omega history topic 10: In honor of #KubeConEU and the 5th anniversary of open-sourcing Kubernetes, I’ll add more perspective from the Borg and Omega teams to the origin story
Show this thread
-
Internally, Google puts a lot of emphasis on both resource efficiency and engineering efficiency. For both reasons, back in June 2013, a few months before GCE was ready to GA (https://cloudplatform.googleblog.com/2013/12/google-compute-engine-is-now-generally-available.html …), the Borg and GCE teams started to work more closely to improve both
Show this thread
The initial focus gravitated towards directly supporting features needed by Cloud in Borg so that Cloud wouldn't need to work around the the lack of those features (see my previous comment on 2-level scheduling: https://twitter.com/bgrant0607/status/1126153859041087488 …)
-
-
Google also spends a lot of energy constantly mitigating entropy in its internal software and infrastructure. The monorepo is one mechanism for this. It also launches many efforts to “unify” or “converge” multiple systems that co-evolve to do similar things.
Show this thread -
In that context, 2 months later, the Unified Compute Working Group was formed by Google Cloud and Google's internal infrastructure group, "TI", which included Borg. The goal was to develop a proposal for a "compute platform" that could be used by both Cloud and internal customers
Show this thread -
It was obvious that VMs would be too cumbersome and inefficient and PaaSes of the time weren't versatile enough to run a wide range of internal services, such as web search and Gmail. We needed a platform that was more like Borg, that was based on containers.
Show this thread -
There were discussions regarding how compatible it should be with App Engine and with Borg. Docker, buildpacks, and Omlet (a new node agent under development to replace Borglet) were compared. Early discussions presumed a managed service, like GCE, App Engine, and Borg
Show this thread -
In September 2013, viewpoints of 9+ WG participants were collected and composed into a "Unified Compute PRD", focused on serving workloads (e.g., rather than batch). That was the first time I was aware of the term "Container as a Service" being used
Show this thread -
In October, subgroups of the WG were formed to focus on key problems, including a container management API subgroup. In November, we pulled in more people from Borg and from Cloud to hash out a number of API details. In December, an API proposal was presented to the full WG
Show this thread -
At the same meeting, a proposal was made for what became the App Engine Flexible Environment (https://cloud.google.com/appengine/docs/flexible/ …), and a proposal to build an open-source container platform, so that we wouldn't be "Hadooped" by other OSS projects
Show this thread -
That OSS container platform was Project 7. After, there were several proposals from both the Borg side and Cloud side to build products with compatible APIs. Collaboration with the Borg team deepened. Borglet team members started to work on libcontainer for Docker in April 2014
Show this thread -
It soon became clear that other Borg team members (me
@thockin@erictune4 Dawn Chen@originalavalamp@davidopp@vishnukanan) should work on the open-source project to design and develop the Borg-like functionality. We deeply believed in the potential value to external usersShow this thread -
So Google Cloud created Kubernetes in part because Google needed it, and we believed others needed it also. Looking at the other solutions available at the time (e.g., https://github.com/tsuru/docker-cluster …, https://github.com/signalfx/maestro-ng …) and since, we made the right call
Show this thread -
Ah, I found that
@jbeda covered some of this background from the Cloud side in his podcast: https://kubernetespodcast.com/episode/012-kubernetes-origins/ …Show this thread -
Dawn spoke about her involvement in her podcast: https://kubernetespodcast.com/episode/022-sig-node/index.html …
Show this thread -
And other previous origin stories and retrospectives: https://cloudplatform.googleblog.com/2016/07/from-Google-to-the-world-the-Kubernetes-origin-story.html … https://kubernetes.io/blog/2016/07/happy-k8sbday-1/ … https://kubernetes.io/blog/2016/07/the-bet-on-kubernetes/ … https://kubernetes.io/blog/2016/07/oh-the-places-you-will-go/ …https://kubernetes.io/blog/2018/06/06/4-years-of-k8s/ …
Show this thread -
There were some posted after the OSCON Most impact award also: https://kubernetes.io/blog/2018/07/19/kubernetes-wins-2018-oscon-most-impact-award/ … https://cloudplatform.googleblog.com/2018/07/kubernetes-wins-oscon-most-impact-award.html …https://kubernetes.io/blog/2018/07/20/the-history-of-kubernetes-the-community-behind-it/ …
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.